Skip to content

fix: rewrite double-quoted DuckDB sources in WASM#9925

Merged
mscolnick merged 1 commit into
marimo-team:mainfrom
peter-gy:ptr/wasm-duckdb-ast-scan-fix
Jun 18, 2026
Merged

fix: rewrite double-quoted DuckDB sources in WASM#9925
mscolnick merged 1 commit into
marimo-team:mainfrom
peter-gy:ptr/wasm-duckdb-ast-scan-fix

Conversation

@peter-gy

@peter-gy peter-gy commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Follow up of #9480.

Fixes DuckDB WASM SQL rewrites for read_csv("https://...") and other read_* functions. DuckDB accepts double-quoted file sources, but sqlglot represents them as quoted identifiers not string literals, so we skipped the WASM remote-source rewrite and raised.

This PR normalizes DuckDB string-like reader arguments before source validation.

Before

Screenshot 2026-06-18 at 18 14 53

After

Screenshot 2026-06-18 at 18 15 04

Review in cubic

Copilot AI review requested due to automatic review settings June 18, 2026 16:22
@vercel

vercel Bot commented Jun 18, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
marimo-docs Ready Ready Preview, Comment Jun 18, 2026 4:23pm

Request Review

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes DuckDB WASM SQL remote-source rewrites when DuckDB readers (e.g. read_csv) are called with double-quoted URL arguments that sqlglot parses as quoted identifiers/columns rather than string literals. This ensures remote URL fetching/replacement-scan rewrites trigger correctly in Pyodide/WASM notebooks.

Changes:

  • Normalize DuckDB “string-like” reader arguments by treating quoted identifiers/columns as string literal values for WASM source detection.
  • Reuse the same normalization for option parsing so double-quoted string options don’t get rejected as dynamic.
  • Add a regression test covering read_csv("https://...") rewrite behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
marimo/_runtime/_wasm/_duckdb/sources.py Adds _string_literal_value() and uses it for reader source + option literal extraction, enabling double-quoted URL rewrites under sqlglot.
tests/_runtime/test_duckdb_wasm.py Adds a unit test asserting wrapped_sql rewrites read_csv("https://...") and triggers the WASM fetch path.

@peter-gy peter-gy requested a review from mscolnick June 18, 2026 16:25

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Architecture diagram
sequenceDiagram
    participant Test as UserTest
    participant Wrapped as wrapped_sql()
    participant Parser as SQLParser (sqlglot)
    participant Rewriter as RewriteEngine
    participant StringLit as _string_literal_value (NEW)
    participant Fetch as fetch_url_bytes
    participant DuckDB as DuckDB Connection

    Test->>Wrapped: call wrapped_sql(read_csv("https://..."))
    Wrapped->>Parser: parse SQL to AST
    Parser-->>Wrapped: AST with read_csv Function
    Wrapped->>Rewriter: rewrite_remote_sources(AST)
    Rewriter->>Rewriter: find read_* functions
    Rewriter->>Rewriter: _read_function_source(args[0])

    Note over Rewriter: arg is exp.Column (double-quoted identifier)
    Rewriter->>StringLit: _string_literal_value(exp.Column)
    alt Identifier without table/db/catalog
        StringLit->>StringLit: check quoted identifier
        StringLit-->>Rewriter: "https://datasets.marimo.app/cars.csv"
    else old behavior (Identifier not Literal)
        Rewriter-->>Rewriter: would return None → raise error
    end

    Rewriter->>Rewriter: source is remote (https)
    Note over Rewriter: NEW: rewrite to use fetch_url_bytes
    Rewriter-->>Wrapped: rewritten SQL (fetches URL)
    Wrapped->>Fetch: fetch_url_bytes(url)
    Fetch-->>Wrapped: CSV bytes
    Wrapped->>DuckDB: execute(SELECT FROM read_csv_auto(bytes))
    DuckDB-->>Wrapped: relation
    Wrapped-->>Test: relation rows

    Note over Test,Fetch: Test asserts fetch_url_bytes called once
Loading

Re-trigger cubic

@mscolnick mscolnick merged commit 5405b64 into marimo-team:main Jun 18, 2026
29 of 44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants